Kernel-based Mixture Models for Classification

Kernels are now everywhere present in statistics as far as a dot product is at hand. However to the best of our knowledge kernels have not been used in mixture models. In the present work we show that they can be useful for classification purposes. They offer a flexibility in the modeling process through the kernel trick which enables capturing interesting features in some cases more easily than just using the standard methods. Our method is based on mixtures of Gamma distributions. These model the point distances to cluster centroids in the transformed space. The distances are readily computed using the kernel trick. Nested within our model are the two special cases of a mixture of exponentials and the kernel K-means method. We suggest using the log-likelihood ratio or the Bayesian Information criterion to select an appropriate parsimonious model for the data at hand. A comparison with other popular classification methods such as support vector machines, shows that our method is very competitive and computationally efficient.